1. Field of the Invention
The present invention relates to a system and a method for rectifying stereo images in real-time. More particularly, the present invention relates to a system and a method for rectifying stereo images, which are acquired by two cameras, in real-time by using a calibration matrix resulting from camera calibration.
2. Description of the Prior Art
In general, when a stereo vision operation is conducted by using two cameras, stereo matching for finding corresponding points from two images is one of the most challenging issues.
Images acquired by two cameras are not parallel to each other in general, and, in order to explain the relationship between both cameras with regard to the acquired images, the concept of a baseline, an epipole, and an epipolar line must be defined.
The baseline refers to a line joining the centers of two cameras; the epipole refers to a point of intersection between the baseline and an image plane; and the epipolar line refers to a line joining an image point to the epipole.
In order to find corresponding points from two images A and B, which have different epipolar lines, one point of image A must be compared with every point of image B (i.e. two-dimensional searching), in theory. This requires a large amount of calculation and renders no real-time operation possible.
Therefore, in order to find identical points from two images in real-time, their epipolar lines must be made to coincide so that the searching range is reduced to one dimension.
To this end, it is conventional to obtain image conversion matrices based on characteristic values resulting form camera calibration and apply the matrices to respective images acquired by cameras.
This process involves complex operations, including multiplication and inversion of 3×3 matrices. As a result, real-time implementation becomes difficult, and the overall system complexity increases.
In an attempt to shorten the operation time, operation results may be pre-stored for respective pixels and referred to as a lookup table when necessary. However, this approach cannot be applied flexibly when camera parameters (e.g. relative location, focusing distance) vary at a later time.
Technologies regarding camera calibration are disclosed in U.S. Pat. Nos. 6,437,823, 6,101,455, and 6,985,175.
These technologies, which are mainly focused on camera calibration, propose systems and methods for calibration and rectification regardless of speed.
However, the conventional technologies disclosed in the above-mentioned patent publications are hardly applicable to industrial fields (e.g. robots) due to the limitation that they use universal computers or structured light.